1 research outputs found
Image sensing with multilayer, nonlinear optical neural networks
Optical imaging is commonly used for both scientific and technological
applications across industry and academia. In image sensing, a measurement,
such as of an object's position, is performed by computational analysis of a
digitized image. An emerging image-sensing paradigm breaks this delineation
between data collection and analysis by designing optical components to perform
not imaging, but encoding. By optically encoding images into a compressed,
low-dimensional latent space suitable for efficient post-analysis, these image
sensors can operate with fewer pixels and fewer photons, allowing
higher-throughput, lower-latency operation. Optical neural networks (ONNs)
offer a platform for processing data in the analog, optical domain. ONN-based
sensors have however been limited to linear processing, but nonlinearity is a
prerequisite for depth, and multilayer NNs significantly outperform shallow NNs
on many tasks. Here, we realize a multilayer ONN pre-processor for image
sensing, using a commercial image intensifier as a parallel optoelectronic,
optical-to-optical nonlinear activation function. We demonstrate that the
nonlinear ONN pre-processor can achieve compression ratios of up to 800:1 while
still enabling high accuracy across several representative computer-vision
tasks, including machine-vision benchmarks, flow-cytometry image
classification, and identification of objects in real scenes. In all cases we
find that the ONN's nonlinearity and depth allowed it to outperform a purely
linear ONN encoder. Although our experiments are specialized to ONN sensors for
incoherent-light images, alternative ONN platforms should facilitate a range of
ONN sensors. These ONN sensors may surpass conventional sensors by
pre-processing optical information in spatial, temporal, and/or spectral
dimensions, potentially with coherent and quantum qualities, all natively in
the optical domain